Gated Recurrent Context: Softmax-Free Attention for Online Encoder-Decoder Speech Recognition

Recently, attention-based encoder-decoder (AED) models have shown state-of-the-art performance in automatic speech recognition (ASR). As the original AED with global attentions are not capable of online inference, various attention schemes been developed to reduce ASR latency for better user experience. However, a common limitation conventional softmax-based approaches is that they introduce an additional hyperparameter related length window, requiring multiple trials model training tuning hyperparameter. In order deal this problem, we propose novel softmax-free method and its modified formulation attention, which does need any at phase. Through number experiments, demonstrate tradeoff between proposed technique can be controlled by merely adjusting threshold test Furthermore, methods showed competitive terms word-error-rates (WERs).